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Abstract 

Pragmatic comprehension can be ideally developed through contact with target language speakers. This contact 
can be provided in English as Foreign Language contexts through telecollaboration. To test the actual effect of 
telecollaboration on the development of pragmatic comprehension, 30 Iranian undergraduates of English as a 
Foreign Language participated in an experimental study. The participants were divided into two groups of 15 
students: control group which merely received pragmatic instructions and experimental group which had the 
opportunity to interact with target language speakers through internet-mediated communication tools besides 
receiving pragmatic instructions. The data were collected through a multiple choice pragmatic comprehension 
test, following a semester-long intervention. The comparison of the performance of the two groups assessed 
through the independent-samples t-test showed the significant positive effect of telecollaboration on the 
development of pragmatic comprehension. Pedagogical implications of the findings suggested organizing 
telecollaborative partnership environments to link English as Foreign Language students with university students 
in native English speaking countries. 
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1. Introduction 

Pragmatic comprehension involves understanding the meaning of expressions at two levels: assigning sense to 
the words uttered and the speaker’s intention behind the words (Thomas, 1995). To be able to comprehend the 
intended meaning of a target language expression, the semantic aspect of the expression (what is communicated) 
needs to be complemented with the pragmatic aspect of the expression (what is meant) (Holtgraves & Kashima, 
2008). The reason is the intended meaning of expressions in a given context might differ considerably from 
culture to culture, resulting in various interpretations of the same utterance (Murray, 2010). This is attributed to 
the fact that the sociolinguistic perspectives of the source language and the target language differ significantly 
from each other (Alptekin, 2002). 

Therefore, the ideal way to gain pragmatic competence in a certain language is through exposure to target 
language culture and contact with target language speakers (Farashaiyan & Tan, 2012; Rafieyan et ah, 2014; 
Rafieyan et ah, in press). Such an environment can be provided through educational sojourns, “periods spent 
abroad in a region where a target language is used as a medium of everyday communication” (Culhane, 2004, p. 
50). However, educational sojourns cannot be afforded by the majority of language learners. Thus, recognizing 
the essentiality of exposure to target language culture to develop pragmatic comprehension, on one hand, and 
considering the costs of educational sojourns, on the other hand, calls for organizing cost-effective programs for 
language learners to have frequent contacts with target language speakers. 

Such an intercultural contact can be conveniently facilitated through the provision of opportunities for foreign 
language learners to engage in computer-mediated intercultural communication with target language speakers 
using technological tools such as synchronous chat and e-mail as well as popular social networks such as 
Facebook and Skype through an approach referred to as “telecollaboration”. Telecollaboration, defined as 
“institutionalized, electronically mediated intercultural communication under the guidance of a languacultural 
expert (i.e., a teacher) for the purposes of foreign language learning and the development of intercultural 
competence” (Belz, 2003, p. 2), involves the application of global computer networks to foreign language 
learning in institutionalized setting. 
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In telecollaborative partnership, internationally dispersed language learners in parallel language classes are 
provided with cost-effective access to, and engagement with, peers who are expert speakers of the language 
under study (Belz, 2005). Telecollaboration has been shown to be especially conductive to the development of 
target language pragmatic competence as this approach “may expand the variety of discourse options to which 
learners are exposed as well as create opportunities for the performance and practice of target language 
pragmatic competence in meaningful interactions” (Belz, 2007, p. 52). Thus, telecollaboration can provide an 
ideal environment for language learners to experience intercultural contact at home which can compensate for 
the shortcomings of educational sojourns. 

2. Literature Review 

2.1 Telecollaboration 

In a study to examine the effect of telecollaborative partnership on the development of pragmatic competence, 
Kinginger and Belz (2005) focused on a particular feature of pragmatic competence namely address form 
competence in German. The study consisted of an English-speaking learner of German in the United States who 
participated for 8 weeks in an electronically mediated partnership with expert speakers of German through an 
Internet-mediated class-to-class pedagogical exchange. In examining the telecollaborative classroom, a 
corpus-assisted microgenetic approach characterized as “the observation of skill acquisition during a learning 
event” (Belz & Kinginger, 2003, p. 594) with a longitudinal scope was adopted. The findings suggested the 
learner’s development pathway toward expertise in the use of the address form system. 

In another study, Vyatkina and Belz (2006) employed the twin research methodologies of contrastive learner 
corpus analysis and microgenesis in the context of telecollaborative language and culture learning partnerships to 
examine the emergence of a critical feature of pragmatic competence namely the comprehension and use of 
modal particles in German. The participants in the study consisted of 16 American students of German at a 
university in the United States and their 23 German key-pals enrolled at a college in Germany. Telecollaborative 
native speaker - nonnative speaker correspondence lasted for 9 weeks through e-mail and synchronous chat. The 
findings indicated that the experimental design proved to be conductive to the development of pragmatic 
competence with respect to performance. 

Most recently, Cunningham and Vyatkina (2012) conducted a study over the effect of telecollaborative 
partnership on the development of pragmatic competence. More specifically, they investigated whether 
interaction with expert users of German combined with a data-driven instructional intervention improve German 
learners’ use of politeness strategies. The participants in their study consisted of 9 American students of German 
at a university in the United States. The instructional context of the study was a telecollaborative web 
conferencing exchange between learners of German and German professionals. The study utilized the method of 
microgenetic analysis. The findings of the study indicated that instruction proved to be conductive to the 
development of pragmatic competence. 

2.2 Pragmatic Comprehension 

In one study, Taguchi (2008) investigated whether the ability to comprehend implied meanings, defined as 
pragmatic comprehension, develops through the passage of time and whether the gains in pragmatic 
comprehension are associated with the amount of target language contact outside the class. The participants were 
44 Japanese students of English as a Second Language at a college in the United States. Pragmatic 
comprehension ability was evaluated through a yes/no test assessing comprehension of implied meanings. A 
language contact survey instrument was also administered to document the amount of outside class contact. The 
findings suggested that pragmatic comprehension improved over time. Furthermore, the amount of language 
contact correlated significantly with gains in pragmatic comprehension ability. 

Alagozlu and Buyukozturk (2009) also conducted another study to investigate the ability to comprehend 
pragmatically implied meanings and to explore the relationship between pragmatic comprehension ability and 
linguistic knowledge. The participants in the study consisted of 25 students of English as a Foreign Language at 
a university in Turkey. Pragmatic comprehension ability was measured through a multiple choice test assessing 
comprehension of implied meanings. Linguistic knowledge was also assessed through an English language 
proficiency test. The results of the study suggested the relative low degree of pragmatic comprehension ability 
compared to other aspects of linguistic knowledge. Furthermore, the results showed no relationship between 
pragmatic comprehension ability and linguistic knowledge. 

In another study, Manowong (2011) examined the ability to comprehend pragmatically implied meanings in 
Engish as a Foreign Language as well as the factors which underlie the ability to understand implied meanings. 


12 




www.ccsenet.org/elt 


English Language Teaching 


Vol. 7, No. 2; 2014 


The participants in the study consisted of 40 Thai students of English as a Foreign Language at a university in 
Thailand. The ability to interpret pragmatically implied meanings was assessed through a multiple choice test 
measuring comprehension of implied meanings. A think-aloud procedure was also conducted to obtain 
information about the process of making inferences as well as the factors underlying the interpretation. The 
results of the research indicated that lack of linguistic competence and cultural competence led to language 
learners’ poor interpretation of pragmatically implied meanings. 

3. Methodology 

Studies over the effect of telecollaborationin on developing pragmatic competence conducted to date have 
dominantly explored the production aspect of pragmatic competence, specifically focusing on politeness 
strategies. Given the benefits of telecollaboration in facilitating contact with target language speakers and 
exposure to target language culture as predictors of pragmatic comprehension development on one hand and the 
significance of possessing pragmatic comprehension knowledge to comprehend the intended meaning of target 
language texts and expressions on the other hand, the current study sought to explore the effect of 
telecollaborative partnership on the development of pragmatic comprehension in an English as a Foreign 
Language context. 

Specifically, the research question is: 

Does telecollaboration have a role in developing Iranian English as Foreign Language students’ level of 
pragmatic comprehension? 

Accordingly, the he null hypothesis is: 

Telecollaboration has no role in developing Iranian English as Foreign Language students’ level of pragmatic 
comprehension. 

3.1 Participants 

The participants in the study were 30 senior undergraduate students of English as a Foreign Language in the 
University of Tehran, Iran. The ages of participants were between 21 and 24, with a mean of 22. Among the 30 
participants in the study, 12 were males and the remaining 18 were females. The participants had passed several 
courses in English grammar, reading comprehension, and conversation during the first three years of their 
undergraduate studies. Thus, they were considered to possess an equally sufficient linguistic competence in 
English. Also, according to the demographic data collected through the demographic questionnaire, none of the 
participants had experienced visiting or living in a native English speaking country nor had native English 
speaking friends or pen-pals. Therefore, they did not have the opportunity to develop pragmatic competence. 

Besides the 30 participants in the study, 12 American students in the University of Texas participated in the 
study to provide target language contact for Iranian participants. The students in the University of Texas were 
also at the undergraduate level. Their ages ranged from 22 to 28 years old, with an average age of 25 years old. 
Among all of the students, 5 were males and the remaining 7 were females. Furthermore, the selection of these 
students for the study was based on their voluntary participation. Therefore, as they were approximately at the 
same age and study level as the focal participants in the University of Tehran and also had a strong tendency to 
interact with the focal participants, the study could ideally achieve its aim of providing continuous and first-hand 
contact with native English speakers for the students of English as a Foreign language at their home country. 

3.2 Instruments 

A demographic questionnaire and a pragmatic comprehension test were used as data collection instruments for 
the present study. The demographic questionnaire contained questions eliciting information related to students’ 
age, visits to English speaking countries, and contact with native English speakers. The pragmatic 
comprehension test was a 30-item multiple choice pragmatic listening comprehension test developed by Taguchi 
(2007, 2008). The test consisted of two types of items: 24 experimental items and 6 literal items. Experimental 
items tested comprehension of implied opinions while literal items tested literal comprehension. Literal items 
were used merely as distractors. Each item on the test contained a conversation between a female and a male 
native English speaker followed by four choices including one appropriate answer and three distractors. 

The validity of the pragmatic comprehension test was assessed through content-related evidence of validity. The 
researchers wrote out the definition of what they wanted to measure and gave the definition, along with the test 
and a description of the intended sample, to two lecturers who were experts in the field of pragmatics at the 
Universityof Tehran to judge the appropriateness of the items to be used for the current study. Both lecturers 
agreed on the appropriateness of the test for the study. The reliability of the pragmatic comprehension test was 
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also assessed through split-half reliability coefficient. Prior to the main study, the pragmatic comprehension test 
was piloted over 20 non-participant senior undergraduates of English as a Foreign Language in the University of 
Tehran. The split-half reliability coefficient assessed for the pragmatic comprehension test was 0.80. 

3.3 Procedure 

At the first week of the winter semester 2013, the 30 Iranian students participating in the study were randomly 
assigned to two equal groups of 15 students: an experimental group and a control group. Therefore, the study 
enjoyed a true experimental design by random assignment of participants to the two groups (Fraenkel et al., 
2012). The students in the control group received pragmatic instruction along with their regular class instructions. 
Pragmatic instructions were incorporated into their classroom instructions three times weekly, for 20 minutes 
each session. Students in the experimental group also received pragmatic instructions accordingly as the control 
group. However, arrangements were made for them to have direct contacts with American participants during 
their course of instructions in addition to the pragmatic instructions. 

During the intervention, students in the experimental group were paired with the American students. The 
students in both Iran and the United States were kept in touch through Facebook in an educational group 
designed and supervised by the researchers. The theme of their interactions was the culture-specific behaviors of 
Americans. American students were required to explain their cultural beliefs and behaviors regarding the specific 
issues listed by the researchers. Iranian students were also required to express their perceptions toward those 
culture-specific behaviors and seek their American peers to comment on their perceptions. Students in both 
countries also had to compare the general views of Americans and Iranians regarding each culture-specific 
behavior. Interactions were set to be conducted within one specific hour, convenient to both sides, twice weekly. 

The pragmatic instructions, which were allocated to both control group and experimental group, as well as the 
contact with native English speakers, which was allocated to only the experimental group, were all continued for 
the whole winter semester for 18 weeks starting from February 19 to June 28. Following the instructions and 
interactions, at the end of the semester, the Iranian students were tested on their comprehension of implied 
opinions through the administration of the pragmatic listening comprehension test to all students in both groups. 
Following the completion of the pragmatic comprehension test, 30 copies of the demographic questionnaire were 
also distributed among the 30 students to collect demographic information related to the participants. The 
questionnaire and the test were then collected by the research team for the purpose of data analysis. 

3.4 Data Analysis 

To evaluate the performance of English as Foreign Language students in both the control group and the 
experimental group on pragmatic comprehension test, the 6 items testing students’ comprehension of literal 
meanings were excluded from the analysis since they were merely used as distractors to distract students’ 
attentions from the main purpose of the pragmatic comprehension test which was testing their ability to 
comprehend pragmatically implied opinions. Then each correct answer on the test was assigned 1 mark while no 
mark was allocated to wrong answers. As there were 24 experimental items on the pragmatic comprehension test 
(after excluding the 6 distractor items from the 30 items included in the pragmatic comprehension test), each 
student could get a mark of between 0 and 24. 

To compare the performance of the students in the experimental group with those in the control group, the mean 
scores for students’ marks in each group was computed. The mean scores of both control group and experimental 
group were then compared using independent-samples t-test, which is applied in case the aim is comparing the 
mean score for two different groups of participants (Pallant, 2013). In the analysis of independent-samples t-test, 
the significance value was adjusted at 0.05. Accordingly, the difference in the scores obtained from the control 
group and the experimental group would be considered significant if the probability value was less than or equal 
to 0.05 (p < 0.05). The difference would be also considered non-significant if the probability value was greater 
than 0.05 (p > 0.05). The analysis was performed using SPSS, version 20. 

To measure the size of the difference in the mean scores obtained by the control group and the experimental 
group, effect size was measured. The measure of effect size used for the current study was the percentage of 
variance accounted for, r 2 (r squared) which is a commonly used method of measuring effect size for 
independent-samples t-test. r 2 measures how much of the variability in the scores can be explained by the 
treatment effects (Gravetter & Wallnau, 2013). The formula to measure r squared is “r 2 = t 2 /1 2 + df’. The criteria 
for interpreting the value of r 2 , as proposed by Cohen (1988), are presented in table 1. According to this table; a 
value of 0.01 indicates a small effect size, a value of 0.09 indicates a medium effect size, and a value of 0.25 
indicates a large effect size. 
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Table 1. Percentage of variance explained, r 2 

r 2 = 0.01 

Small Effect 

r 2 = 0.09 

Medium Effect 

r 2 = 0.25 

Large Effect 


4. Results and Discussion 

4.1 Results 

4.1.1 Descriptive Data 

Table 2 presents descriptive data consisting of the mean and the standard deviation obtained from the English as 
Foreign Language students’ performance on the pragmatic comprehension test. The data have been presented for 
both groups including the control group and the experimental group. The table also presents the number of 
students allocated to each group participating in the current experimental study. According to this table, there are 
an equal number of 15 participants in each group. The mean score and the standard deviation obtained for the 
students’ performance on the pragmatic comprehension test in the control group are respectively 9.33 and 2.09. 
The mean score and the standard deviation obtained for the students’ performance on the pragmatic 
comprehension test in the experimental group are also respectively 18.00 and 3.07. 


Table 2. Group statistics 



Groups 

N 

Mean 

Std. Deviation 

Std. Error Mean 

Pragmatic 

Control Group 

15 

9.33 

2.09 

0.54 

Comprehension Level 

Experimental Group 

15 

18.00 

3.07 

0.79 


4.1.2 Assumption for Independent-Samples T-Test 

There are a number of assumptions underlying independent-samples t-test that should be satisfied. The major 
assumption among all of the assumptions is “homogeneity of variance” which states that the two populations 
being compared must have the same variance (Gravetter & Wallnau, 2013). SPSS uses Levene’s test for equality 
of variances to assess homogeneity of variance. The results of Levene’s test for equality of variances to assess 
the homogeneity of variance for the two groups participating in the current study have been presented in table 3. 
Levene’s test for equality of variances tests whether the variation of scores for the students in the control group 
and the experimental group is the same or not (Pallant, 2013). The outcome of this test determines which of the 
t-values that SPSS provides is the correct one for the current study to use. 


Table 3. Independent samples test 




Levene’s 
Test of 
Equality of 
Variances 



t-test for Equality of Means 





F 

Sig. 

T 

df 

Sig. 

(2-tailed) 

Mean 

Diffe 

-ence 

Std. 

Error 

Differ 

-ence 

95% Confidence 
Interval of the 
Difference 

Lower Upper 


Equal 










Pragmatic 

Variances 

1.51 

0.23 

-9.03 

28.00 

0.00 

-8.67 

0.96 

-10.63 

-6.70 

Comprehen¬ 

Assumed 










sion Level 

Equal 











Variances not 



-9.03 

24.70 

0.00 

-8.67 

0.96 

-10.64 

-6.69 


Assumed 
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If the significance value for Levene’s test is larger than 0.05 (p > 0.05), the first line of this table, which refers to 
“Equal Variances Assumed”, should be used. If the significance level of Levene’s test is less than or equal to 
0.05 (p < 0.05), the variations for the control group and the experimental group are considered not to be the same 
and the data violate the assumption of equal variance. Therefore, the information in the second line of the t-test, 
which refers to “Equal Variances not Assumed”, should be used (Pallant, 2013). According to this table, the 
significance level for Levene’s test is 0.23. This is larger than the cut-off of 0.05. This means that the assumption 
of equal variances has not been violated. Therefore, the information provided in the first line of the table, 
referring to “Equal Variances Assumed”, will be used to report the t-value. 

4.1.3 Differences between the Groups 

To find out whether or not there is a significant difference between the performance of students in the control 
group and the experimental group on the pragmatic comprehension test, the significance value will be used. A 
significance value of equal to or less than 0.05 (p < 0.05) indicates a significant difference in the mean scores on 
the pragmatic comprehension test for the control group and the experimental group, whereas a significance value 
of above 0.05 (p > 0.05) indicates an insignificant difference between the two groups (Pallant, 2013). According 
to the above table, the significance value for the performance of the two groups on the pragmatic comprehension 
test is 0.00. As this value is below the cut-off of 0.05, there is a statistically significant difference in the mean 
scores for the students’ performance in the control group and the experimental group. 

4.1.4 Effect Size 

To measure the effect size, the values obtained for “t” and “df’, provided in table 3, were put into the formula 
used to compute r 2 . Putting the values into the formula would give the equation “r 2 = (-9.03) 2 / (-9.03) 2 + 28” 
which produces a value of 0.74 for r 2 . Therefore, the effect size computed through r 2 for the size of the difference 
in the mean scores obtained by the control group and the experimental group in the current study is 0.74. This 
value, according to the standard used to evaluate r 2 by Cohen (1988), indicates a very large effect size. Expressed 
as a percentage, 74 percent of the variation in the pragmatic comprehension scores is explained by the type of 
treatment. In other words, type of treatment has a very large effect on the development of level of pragmatic 
comprehension in English as Foreign Language students. 

4.1.5 Summary of the Findings 

To summarize the findings obtained from the current study, the students in the experimental group, who had 
contact with native English speakers along with pragmatic instructions during the course of instructions, had 
higher marks (mean: 18.00, standard deviation: 2.09) on pragmatic comprehension test than the students in the 
control group, who merely received pragmatic instructions without having contact with native English speakers 
(mean: 9.33, standard deviation: 3.07). The magnitude of difference in the mean scores obtained by the control 
group and the experimental group was extremely significant, t (-9.03) = -8.67, p < 0.05, r 2 = 0.74. These findings 
reject the null hypothesis posed for the current study which stated telecollaboration has no role in developing 
Iranian English as Foreign Language students’ level of pragmatic comprehension. 

4.2 Discussion 

The results of this research study suggested that contact with target language speakers had a remarkable effect on 
Iranian English as Foreign Language students’ pragmatic comprehension development. The participants in the 
experimental group, who had direct contact with native English speakers along with the pragmatic instructions 
which they received during their course of study, scored much higher on the pragmatic comprehension test than 
the participants in the control group, who merely received pragmatic instructions without having contact with 
native English speakers. This clearly signifies the influential role of first-hand contact with target language 
speakers and insufficiency of pragmatic instructions in English as Foreign Language contexts in the development 
of pragmatic comprehension. 

There are a number of factors which can explain the reason behind the superiority of contact with target 
language speakers compared to the mere instruction of pragmatic competence in developing pragmatic 
comprehension. The first factor is that textbooks which are used to teach target language pragmatic knowledge 
contain conversational models which are not naturally evident in the target language interactions (Nguyen, 2011). 
For instance, a review of instructional materials for language learners reveals that textbooks normally mention 
the expression “I disagree with ...” in order to provide models of disagreement in the target language context. 
Flowever, observation of native speakers’ interactions indicate that the expression “well ... but ...” is much more 
frequently used than the ones mentioned in textbooks (Boxer & Pickering, 1995). 
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The other factor underlying the superiority of contact with target language speakers compared to pragmatic 
instructions on the development of target language pragmatic comprehension is that the ideal way of obtaining 
the pragmatic knowledge of a certain language is through exposure to target language culture and contact with 
target language speakers (Farashaiyan & Tan, 2012; Rafieyan et al., 2014; Rafieyan et ah, in press). Students in 
the experimental group were exposed to the sociolinguistic features of the target language community through 
constant interactions with the target language people and consequently had more opportunities to pick those 
sociolinguistic features while students in the control group were deprived from such exposure to target language 
culture and interaction with target language people to develop their target language pragmatic comprehension. 

The final factor which might explain the superior performance of students in the experimental group can be 
attributed to the motivational factors. Motivation can be the most influential variable to derive language learners’ 
attention to notice the sociolinguistic perspectives of the target language society (Takahashi, 2001; Rafieyan et 
ah, 2013a, 2013b, 2013c). Contact with target language speakers can be a determining factor to provoke 
motivation toward exploring target language culture for English as Foreign Language students. In the current 
research study, students in the experimental group possessed the opportunity to interact with native English 
speakers. Therefore, they were encouraged to explore target language culture. However, there was not such 
stimulating factor to provoke such motivation in students in the control group. 

The findings obtained in this study seem to confirm the findings of previous studies which proved the influential 
role of contact with target language speakers and telecollaborative partnership on the development of target 
language pragmatic competence. These findings support the findings obtained in the study by Taguchi (2008) 
who found that the amount of language contact with target language speakers led to remarkable gains in 
language learners’ pragmatic comprehension ability. The findings of this study are also consistent with the 
findings obtained in the studies by Kinginger and Belz (2005), Vyatkina and Belz (2006), and Cunningham and 
Vyatkina (2012) who found that telecollaborative partnership was conductive to the development of pragmatic 
competence in language learners. 

5. Conclusion 

The current research study found that contact with target language speakers in the form of telecollaborative 
partnership has a significantly positive effect on the development of pragmatic comprehension in English as 
Foreign Language contexts. Although all participants in the study received pragmatic instructions, Iranian 
English as Foreign Language students who had intercultural interactions with native English speakers through 
social networks, referred to as telecollaboration, performed much better on the pragmatic comprehension test 
than those who did not have interaction with native English speakers. This clearly signifies the superiority of the 
role of contact with target language speakers over mere incorporation of target language pragmatic materials into 
the classroom instructions. 

Therefore, considering the costs and shortcomings of providing face-to-face contact with target language 
speakers in the form of educational sojourns, on one hand, and the proved successful outcomes of 
telecollaborative partnership on the development of pragmatic comprehension, on the other hand, suggest the 
provision of telecollaborative partnership environments for the students of English in foreign language contexts. 
Consequently, university deans in English as Foreign Language contexts should make arrangements for their 
English as Foreign Language students to have intercultural interactions with university students in English 
speaking countries through organizing telecollaborative partnership environments to link students in both 
countries together. 
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